High dynamic range image sensors are desirable in a variety of application. As is well known in the field of image sensors, the dynamic range is the ratio of the largest detectable signal to the smallest (which for a CMOS image sensor is often defined by the ratio of the largest non-saturating signal to the standard deviation of the noise under dark conditions). That is, an image sensor's total electrical dynamic range is limited by: 1) the charge saturation level at the upper end; and 2) the noise level at the lower end produced in the analog circuits and A/D conversion. The light dynamic range of a scene is also defined as the ratio between the brightest and darkest objects that can be detected. A high dynamic range (HDR) image sensor may, for example, have a dynamic range greater than 70 dB, such as a dynamic range of 80-100 dB and will typically need more than 12-bits per channel when encoded in a linear space. When the electrical dynamic range of an image sensor is too small to record all light intensity variations in a scene it will result in either the highlight portions being saturated or the shadowed parts being too dark to be recognized. Therefore, there is a desire to increase the dynamic range of an image sensor to accurately reproduce the natural appearance of HDR scenes.
FIG. 1 illustrates a conventional complementary metal oxide semiconductor (CMOS) image sensor. The pixel array 100 has pixels 102 arranged into a set of columns and rows having a column parallel read out architecture in which pixels in a row are read out simultaneously and processed in parallel. That is, Row 0 is read out, then Row 1, then Row 2, and so on until Row M is read out. Sample and hold (S&H) elements support the line-by-line row read out of rows. The rows in a frame have the same exposure time for full resolution modes and down-sampling modes.
The line data that is readout is buffered by a line image buffer. The silicon area (and hence cost) of a line image buffer depends upon the number of lines it must buffer. For many conventional image sensor designs a 3-line image buffer 120 is sufficient to support color interpolation in an image processor 130.
In single sensor color image sensor systems, each pixel on the sensor has a specific color filter determined by the pattern of an array of color filters known as a “color filter array” (CFA). A color image requires at least three color samples in each pixel position. However, a CFA allows only one color to be measured at each pixel. The camera must estimate the missing color values in each pixel. The process is known as color interpolation or demoisaicing. The simplest color interpolation schemes are based on a bilinear interpolation in two dimensions.
FIG. 2 illustrates some of the concepts behind 3-line image buffer bilinear interpolation. Each pixel is assigned a numeric value (e.g., 1, 2, 3, 4, 5) indicative of its location. In this example, a pixel, such as pixel 3, has a color filter such that the pixel samples a color which is not green, such as red or blue. Interpolation is performed for pixel 3 to determine the green (G) channel for pixel 3 based on the green color samples of neighboring green pixels G1, G2, G4, and G5. However, for pixel 3, the green value must be interpolated. G3 is thus an estimated value of the green color in pixel 3. Using bilinear interpolation the estimated green value in pixel 3 is estimated as: G3=(G1+G5+G2+G4)/4. Note that to perform conventional color interpolation on pixel 3 requires data from pixels in the lines above and below the pixel. As can be understood from the example of FIG. 2, conventionally a 3-line image buffer is sufficient to support color interpolation.
A variety of high dynamic range image sensors are known in the prior art. However, many known image sensors have various disadvantages including increased cost for the image sensor and the associated line image buffer memory required to support a high dynamic range. In particular, many previous high dynamic range image sensor approaches have required the use of significantly more expensive hardware and/or buffer memory than for a conventional image sensor.
Therefore, in light of the above described problems the apparatus, system, and method of the present invention was developed.